The present invention relates to digital video signal processing techniques which may be used, for example, in connection with digital television, and more particularly to the coding of different channels of a stereoscopic video sequence.
A stereoscopic video sequence consists of two channels, which are referred to as the left and right channels. The two channels can be coded either independently or jointly. In independent coding the two channels are usually coded at the same bit rate, as there is no particular reason why one channel should be coded with fewer bits than the other channel. The same bit rate likely results in similar quality for the two channels because of the similarity between the channels. However, from the compression point of view, independent coding of the two channels of a stereoscopic sequence is not very efficient because it does not exploit the cross-correlation between two channels.
Disparity estimation as permitted by the Moving Picture Experts Group "MPEG-2 Multi-View Profile" employs the cross-correlation between the two channels. In the MPEG-2 Multi-View Profile, one channel (e.g. the left channel) is considered as a base layer and the second channel (e.g. the right channel) an enhancement layer. The base layer channel is coded independently while the enhancement channel is coded based upon the base layer channel. Simulations have demonstrated a better performance when cross-correlation between the left and right channels is utilized.
MPEG defines three picture types in terms of temporal processing. They are the I (intra-frame coded) picture, P (forward temporal predictive coded) picture, and B (bi-directional temporal predictive coded) picture. The input video sequence is often divided into groups of pictures (GoP). Each GoP contains one I picture, a number of P pictures, and optionally a few B pictures between I or P pictures. I pictures are "intraframe" coded pictures, which are not based on any other pictures (frames) in the sequence. P pictures are "predicted" pictures, which are predicted from previous I or P frames. B pictures are "bi-directionally" predicted pictures, which can be predicted from previous and/or future I and/or P pictures.
Prior to the present invention, schemes for stereoscopic video coding typically used a fixed bit rate for each channel. In general, however, the channel (e.g., the right channel) that is coded based on the other channel (e.g., the left channel) needs fewer bits than the independently coded (e.g., left) channel for similar quality. This is because the non-independently coded (e.g., right) channel has additional information for prediction (e.g., the left channel) and it also has no I pictures which typically use many more bits than P and B pictures.
However, even where one channel is coded based on the other, the optimal sharing of a given channel capacity by the two channels still remains a question. The problem is that the cross-correlation between two channels for different stereoscopic materials can be different, and even within a stereoscopic sequence, the degree of correlation between the two channels can also vary from time to time. Hence, a predetermined bit allocation between two channels which might be quite reasonable for some stereoscopic video sequences, or some segments of a stereoscopic sequence, could be very bad for other sequences, or segments of the same stereoscopic sequence, if equal quality between the two channels is the criterion. Thus, it is difficult to maintain similar quality, or to balance the quality, for the two channels by using a fixed bit allocation strategy for stereoscopic video coding.
It would be advantageous to provide a method and apparatus for jointly controlling the bit rate of the two channels provided for stereoscopic video coding. Such a scheme should allow the bit rates of the two channels to vary according to various criteria, while maintaining the aggregate rate at a constant value within the capacity of a communications channel over which the data are communicated.
The present invention provides a method and apparatus having the aforementioned and other advantages.